THE IDEAL OF THE TRIFOCAL VARIETY 



CHRIS AHOLT^, LUKE OEDING-^ 

Abstract. Techniques from representation theory, symbohc com- 
putational algebra, and numerical algebraic geometry are used to 
find the minimal generators of the ideal of the trifocal variety. An 
effective test for determining whether a given tensor is a trifocal 
tensor is also given. 



1. Introduction 

In the field of multiview geometry one studies n > 2 planar images 
of points in space. Given n full rank 3x4 matrices Ai, . . . , over C, 
these camera matrices determine a rational map 

from projective 3-space into the n-fold product of projective planes. For 
any given tuple (Ai, . . . , An) the image of this map determines a variety 
0(p3) C (p2)" called the multiview variety associated to (Ai, . . . ,v4„). 

In |AST11] the authors determine the prime ideal defining the mul- 
tiview variety for a generic fixed tuple of cameras such that the camera 
matrices Ai, . . . ,y4„ have pairwise distinct kernels. In this paper we 
focus on a different, but related variety in the special case of n = 3 
cameras: the variety of all trifocal tensors |HZ03t Ch. 15]. The essen- 
tial difference is that for the multiview variety the camera matrices are 
fixed and this determines a map from the world to a set of images, but 
in the trifocal setup we consider the set of tensors determined by all 
possible general configurations of triples of cameras. Algebraically, the 
collection of trifocal tensors is parameterized by the 4x4 minors of 
the 4x9 matrix {A\ \ A^ \ A^) which involve one column each from 
the first two blocks, and two columns from the third block |HeyOO| . 
Geometrically, a trifocal tensor arises from a bilinear map describing 
the geometry of a given configuration of cameras. We give a complete 
description of the ideal describing this subvariety of tensors. 

We further describe this geometric map. Each camera matrix Ai 
determines a focal point fi G and a viewing plane vr^ ^ P^ C p3. 
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Figure 1 . The trifocal tensor as a map x P^ 




The image in camera i of a point x G P^ is determined by intersecting 
the hne {ft, x) with the plane vr,. Now consider lines li C tTj for i = 1,2. 
The planes and {f2,h) generically intersect in a line li^2 ^ P^, 

and the plane (/a, /i,2) generically intersects tts in a line I3. See Figure[l] 
We have described for a sufficiently general camera configuration, a 
map 

p2 X p2 ^ P^ 

given by {li,^) ^ h- This map must come from a bilinear map 

C3 X ^ C\ 

To help avoid ambiguity, fix A, 5, C ^ C'^ so that this map is now 
Ax B ^ C. This bilinear map is equivalently a tensor T G A* (8)5* (8>C, 
called a trifocal tensor because of its derivation. For more details, see 
jHZM Chapter 15], [^10], |HeyOO| , jPF98] . 

One way to connect these two seemingly different algebraic and geo- 
metric constructions is via the following construction, which shows (in 
an invariant way) how the parametrization using special minors of a 
4x9 matrix gives rise to a tensor parametrization. This also relates 
to the compactified camera space considered in |ASTllj . 

The row space of (^Aj \ A2 \ A^^ determines a point in the Grass- 
mannian Gr(4, 9). Set Ui, U2, f/3 respectively as the 3-dimensional col- 
umn spaces of the blocked matrix (^Aj \ Al^ \ A3). The direct sum 
W = Ui Q) U2 ® Us is 9-dimensional, and we can view the matrix 
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(^Aj I A2 I A^^ as describing a point in the Grassmannian 

Gr(4,iy) C P(AV)- 

Consider the group G = SL(f/i) x SL(f/2) x SLiU^) C SL{W), which 
can be thought of as the group of (unit determinant) 3x3 blocks on 
the diagonal of a 9 x 9 matrix. Now f^W decomposes as a G-module 
as follows 

f\iu,(BU2(BUs)= (0f/.® A'f^i) © (0A'^^©A'f^. 




If we take A* = Ui,B* = U2 and C* = Us, we see that the factor 
Ui®U2® t\U2, is isomorphic to A* i?* ® C and corresponds to the 
space of maximal minors of a 4 x 9 matrix using 1 column from the 
first two 4x3 blocks and 2 columns from the last 4x3 block. So we 
get a G-equivariant projection 

TT : Gr(4, 9) P {Ui ® f/2 © t\Uz) = F{A* ® ® C), 

and the closure of the image of this projection is the trifocal variety X. 
Because the projection is G-equivariant, the image X is automatically 
G-invariant. The generic fiber of the projection vr is 3 dimensional. 
However, there is a (C*)^ action by scaling each Ai that leaves the 
set of trifocal tensors invariant. The GIT quotient Gr(4, 9)//(C*)^ has 
dimension 18, which is the dimension of X, |ASTlll lATlOj . So if one 
works with the GIT quotient, the map 

vr : Gr(4, 9)//(C*)=^ P (f/i ® t/2 © /^Us) , 

becomes birational to X. 

One would like to know when a given tensor in V = A* ® B* ® C 
arose as a trifocal tensor. The Zariski closure of the set of all such tri- 
focal tensors defines an irreducible algebraic variety, called the trifocal 
variety and hereafter denoted by X C FV. Let I{X) denote the ideal 
of polynomial functions vanishing on X, hereafter called the trifocal 
ideal. Since a tensor T is a trifocal tensor (or a limit of such) if and 
only if T is a zero of every polynomial in the ideal I{X), the question of 
identifying trifocal tensors can be answered (at least for general tensors 
in an open set of X) by determining the minimal generators of I{X). 

In jATlO] the authors determine a set of polynomials that cut out X 
as a set. However, their set of polynomials does not generate the ideal 
/(X). We note that jPF98j and |Res03j also found some equations 
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vanishing on X, but neither described the entire trifocal ideal. The 
focus of this article is to determine the minimal generators of I{X). 

Choosing bases {ai, 02, a^}, {bi, 62, ^3}, and {ci, C2, C3} of A*, B* and 
C respectively, any tensor T G V can be realized as 

3 

T = ^ Tij^kdi (g) bj (g) Cfc 

i,j,k=l 

via the 27 variables Tj j ^ for 1 < j, /c < 3 Therefore, the trifocal ideal 
lives in the polynomial ring k[Tijk]. 

The cubic polynomials in the ideal are the 10 coefficients of 

det(xiTjji + X2Tij2 + XsTijs). 

One component of the zero set of these polynomials is our variety. To 
remove the other components we add polynomials of degree 5 and 6. 

To simplify matters, we will take the ground field to be A; = C; 
however, we note that in practice, one works over M. A tensor with 
real entries is on the complex trifocal variety if and only if it is a zero of 
all polynomials in I{X). And indeed, all of the generating polynomials 
in I{X) can be taken with rational coefficients, and thus are in the 
ideal of X when considered variety over M. 

Our result is the following. 

Theorem 1.1. Let X denote the trifocal variety. The prime ideal I{X) 
is minimally generated by 10 polynomials in degree 3, 81 polynomials 
in degree 5, and 1980 polynomials in degree 6. 

There are noticeably more generators here than in |AT10j . which 
showed that 10 equations of degree 3, 20 of degree 9, and 6 of degree 
12 cut out X set-theoretically. On the other hand, the degrees of our 
equations are lower and we know that they are the minimal degree 
polynomials that generate the ideal. 

2. Outline 



To prove Theorem 1.1 and determine the minimal generators of the 
trifocal ideal, we use a mixture of several different computational and 
theoretical tools that we now outhne. In short, our strategy is to first 
find equations in the ideal in the lowest degrees, next show that the 
equations we found cut out the variety set theoretically and thus define 
an ideal that agrees up to radical with the ideal we want, and then we 
show that the two ideals are actually equal. 

Because the trifocal construction is unchanged by changes of coordi- 
nates in each camera plane, we have a large group G that acts on X. 
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We describe this symmetry and various representations for points on 
X in Section [3j Then we describe the geometry of related G- varieties 
in FV = F{A* ® 5* ® C) in Section |4| 

We let I{X)d denote the degree d piece of I{X), and denote by 
Md{X), (or Md when the context is clear), the (vector space of) minimal 
generators of I{X) occurring in degree d. The group G acts on X and 
thus on I{X). This facilitates the search for all polynomials in I{X)d in 
low degree (for d < 9) via computations using classical representation 
theory. In Section |5] we describe our computations and identify which 
modules of polynomials are minimal generators assisted by symbolic 
computations in Maple and Macaulay2. In particular, we find that the 
only minimal generators of I{X) for d <9 occur in degrees 3, 5 and 6. 
Next, we compute a Grobner basis of the ideal J = (M3 + M5 + Mg), 
and find (again in Macaulay2) that the degree of J is 297. 

Another valuable tool for understanding the zero-set of a collection 
of polynomials is a numerical primary decomposition via numerical 
algebraic geometry. For this we used Bertini |BHSW10] for experiments 
and computations; see also jSWOSj . In Section [g] we consider only M3, 
the lowest degree (degree 3) part of I{X), which has a basis of 10 
polynomials in the 27 variables. Here we obtain a numerical primary 
decomposition of ^(Ms) using Bertini. In particular, we find that up 
to the numerical accuracy of Bertini, ^(Ms) has 4 components, and 
we are even given their degrees. This numerical result provides us with 
tangible data from which we are able to conjecture (and eventually 
prove) the true structure of ^(Ms). In particular, we find that up to 
the numerical accuracy of Bertini, X has degree 297. 

In Section [7] we use geometric considerations and resort to Nurmiev's 
classification of orbits and their closures ( jNurOObl INurOOa] ) to geomet- 
rically identify all the components found by the Bertini computation. 
This geometric understanding allows us to conclude in Proposition |7.4| 
that the zero-set V(J) is equal to X (as sets), so y/j = I{X). In 
Section [8] we again use the classification of orbits and the orbit poset 
structure to show in Theorem 8.5 that J is prime and thus J = I{X). 



3. The trifocal variety as an orbit closure 

Consider V = A* B* ^ C and the natural left action of G = 
SL(A) X SL(fi) X SL{C) ~ SL(3)^3 ji^g^g jg ^^^^ ^ natural 

action of the symmetric group ©3 permuting the three factors in the 
tensor product, and it is easy to see that X is invariant under the action 
of the ©2 permuting A* and B* . However, this finite invariance does 
not provide much computational advantage. 
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Remark 3.1. Since we are working over C we consider general changes 
of coordinates by SL(3, C). However, were we to work over M, we would 
want to change our analysis to consider rotations in the three planes, 
and the group action would be by S0(3,]R)^^. 

Since the trifocal variety X C FV is invariant under changes of coor- 
dinates in the camera planes, we say that X is a G-variety. Moreover, 
I AT 10 ] shows that X is actually the closure of a single G-orbit in FV. 

Because G ~ SL(3)^^ is 24 dimensional acting on \^ ~ C^®C^(8>C^, 
which is 27 dimensional, there must be infinitely many G-orbits in V. 
Even so, the orbits happen to have been classified, apparently indepen- 
dently, by several authors. Since elements of V can be interpreted in 
a number of ways (as triples of 3 x 3 matrices or 3 x 3 matrices with 
linear entries depending on 3 variables, as 3 — 3 — 3 trilinear forms or 
ternary trilinear forms, as cuboids or elements of a triple tensor prod- 
uct, or as a G-submodule of /\^C^), the various classifications occurred 
in different settings — see |TC38l [Ng95| iNuiTinb] . 



We prefer to use Nurmiev's version of the classification, which follow s 



Vinberg's conventions and uses the results and techniques of |VE78 . 



One main reason for this choice is that Nurmiev also computed the 
closures of all the nilpotent orbits in a note |NurOOa] , in the same lan- 
guage as the previous paper. There are 4 continuous families of orbits 
called semi-simple orbits, and one finite family of nilpotent orbits. To 
every orbit O is associated a normal form, which is a representative 
V & V such that G.v = O. Though obviously not unique, we will typi- 
cally choose a normal form that is as simple as possible, or that clearly 
reveals some of the structure of the orbit. 

To use the Nurmiev classification, we first identify the trifocal variety 
as one of the orbits on Nurmiev's list. Indeed, Alzati and Tortora give 
a normal form for the orbit of trifocal tensors that we now recall. A 
general trifocal tensor T may be, after a possible change of coordinates 
by G, identified as a tensor whose slices in the G-direction are 

/O -1 0\ /O 0\ 

= = -1 = 

\i 0/ \o 1 oy 

Choose bases A* = span{ai, 02, 03}, B* = span{6i, 62, ^3}, and G 
span{ci, C2, C3}. Then we can write 

T = (-ai) ® 62 ® ci + (-03) ® (-61) ® ci + (-02) ® 62 ® C2 




+03 ® 62 ® C2 - as ® 62 ® C3 + 03 ® 63 ® C3, 



THE IDEAL OF THE TRIFOCAL VARIETY 



7 



which by changing coordinates (via G) may be written as 

T = ai (g) 62 ® Cl + as (g) 61 (g) Cl + 02 ® &2 ® C2 + as ® &3 ® C3. 

It is also useful to express a tensor T via matrices with linear entries. 
For this, one considers a pure tensor ® bj ® as a matrix with an 
at in the j, k position of the matrix. Then do this for all pure tensors 
in an expression for T and add the matrices. In fact, this describes the 
projections for the P-Rank varieties defined in Section |4j A normal 
form for the trifocal variety has matrices of linear forms 

/as 0\ 0\ /O Cl 0\ 

T{A) = ai a2 , T{B) = 62 , T(C) = C2 . 
\0 as/ \bi 6s/ \ci C3/ 

The difference here is that T{A), T{B), T{C) each individually rep- 
resent T, but the entire set {T^,T^,T^} also represents T. One ad- 
vantage to considering a tensor as a matrix in linear forms is that it is 
now clear that P-Rank(T) = (3,3,2), so X C P-Rank^'^'^ In particu- 
lar, T{C) has rank 2, and thus must satisfy the equations implied by 
det(T(C)) = 0, while T{A) and T{B) do not satisfy this relation. 

Remark 3.2. The construction of the matrix T{A) from the tensor T 
shows that the G action on T corresponds to an action on the matrix 
T{A) by left and right multiplication by elements of SL(i?) and SL(C), 
and by linear changes of variables on the entries of T{A), with similar 
descriptions for the action on T{B) and T{C). 

Nurmiev lists the G-orbits in y as a list of integers. To a triple of 
integers ijk Nurmiev associates the tensor Cj ® ej e^, with 1 < i,j — 
3, k — 6 < 3. The spaces in each expression corresponds to summation. 

For example, consider orbit 11 on Nurmiev's list: 149 167 248 357. 
We choose bases of A*, B* and C so that ai = Cj, bjs = ej, and 
Ck-6 = Gk- So orbit 11 corresponds to the tensor 

fli ® 61 ® cs + fli ® 63 ® Cl + 02 ® 61 ® C2 + as ® 62 ® Cl, 

which corresponds to the matrix of linear forms 

02 ai\ 
as . 
ai / 

Finally, notice that T{C), for instance, can be moved by a change of 
coordinates to 

/O Cl 0\ /O cA /O Cl csX /O cs cA 
T{C) = C2 = C2 = ci = C2 

VCI Cs/ \C1 C3 0/ \C2 0/ \ci 0/ 
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Then swapping the roles of C2 and C3 we obtain 

/O C2 cA 

T{C) ^ C3 . 
\ci 0/ 

This shows that the normal form of T is congruent to orbit 11" on 
Nurmiev's list (where representative 11" is obtained from representative 
11 by performing the permutation a — )■ 6 — )■ c — )■ a twice). 

Nurmiev's list also contains the dimension of the stabilizer of this 
orbit. This confirms for us that the codimension of the trifocal variety 
X is 8 (an already well-estabhshed fact). 

One of the utilities of having a group action is the following. If a 
group G C GL(y) preserves a variety X C FV (i.e. G.X = X), we 
may consider I{X) G-module and in particular G-submodule 
of S*V*, the space of symmetric tensors on V. Recall that S*V* is 
isomorphic to the space of homogeneous polynomials on V. The rep- 
resentation theory of G = SL(y4) x SL{B) x SL(C)-modules is well 
known; however, the reader may wish to consult |Lanl2j or |FH91j for 
reference. One fact we will use is if V = A* ^ B* ^ C, then irreducible 
G-modules in S'V* are all of the form SxA ® 5*^1? (g) S^G*, where X,fi 
and u are all partitions of the same nonnegative integer. 

4. F-Rank and P-Rank varieties 

In the previous section we saw that the matrices T^, T^, represent- 
ing the slices of a trifocal tensor do not have full rank. This condition 
depends on the choice of coordinates. On the other hand, the 3x9 
flattening matrix (T^ | | T^) does have full rank, and this condition 
is not dependent on the choice of coordinates. Of course slicing in a 
different direction may yield a different result, but it is easy to check 
that trifocal tensors have full rank fiattenings for all slices. We refer to 
this condition as flattening rank (F-Ksnik), and note that the general 
trifocal tensor has F-Rank(T) = (3,3,3). 

The matrix T(C) with linear forms in G does not have full rank, 
while the matrices T{A) and T{B) do have full rank. The construction 
of T(C) describes a projection A* ^ B* ^G ^ A* B* , so it is natural 
to refer to the tuple of ranks of the various projections as projection 
rank (P-Rank). A general trifocal tensor T has P-Rank(T) = (3,3,2). 

These two considerations lead to the study of subspace varieties (the 
former) and rank varieties (the latter). Understanding algebraic and 
geometric properties of these varieties will help us find equations for the 
trifocal variety. In what follows we highlight some of these properties. 
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which are specific cases of much more general constructions. For more 
details, see jLanl2l Chapter 7]). 

4.1. Subspace varieties. For p < 3, g < 3, r < 3, the subspace 
variety Subp^g^,. C FV is the projectivization of the set of tensors that 
have F-Rank at most {p,q,r): 

Subp,g,^ = P{r G V I F-Rank(T) < (p,g,r)}, 

where we write (a, b, c) < (p, q,r) if a < p and b < q and c < r. Sub- 
space varieties are irreducible, and their ideals are defined by minors of 
fiattenings (see |LW07t Theorem 3.1]). For the sake of a reader unfamil- 
iar with these concepts, we recall the construction of these equations. 
A tensor T G ^ is realized via 27 variables Tj j- ^ for 1 < i, j, k < 3 

3 

T= ^ Tij^kai^bj ^Ck, 

i,j,k=l 

choosing bases {01,02,03}, {61,62,^3}, and {01,02,03} of A*, B* and 
C respectively. There are three directions in which we may slice T 
to get triples of matrices. Let Wi = {Tij^k)j,k, Yj = {Tij^k)i,k, Zk = 
{Tij,k)i,j denote these slices. Then the matrices W = (Wi \ W2 \ W3) 
— respectively y = (Yi | 1^2 I ^3), and Z = {Zi \ Z2 \ Z3) — are the 
three fiattenings with respect to the three slicings of the tensor T. 

A special case of |LW07l Theorem 3.1] is that the ideals generated by 
the 3-minors of fiattenings are the ideals of subspace varieties. Namely, 

-?'(Sub2,3,3) = {minor s {3, W)), 
/(Sub3,2,3) = {minor s{3,Y)), 
/(Sub3^3 2) = {minor s {3, Z)). 

Moreover, the intersection of two subspace varieties yields another, and 
this holds ideal theoretically as well. Sub2,2,3 = Sub2,3,3 H Sub3^2,3 and 

/(Sub2,2,3) = {minors{3,W)) + {minors{3,Y)), 

and similarly for permutations. Likewise 

/(Sub2,2,2) = {minors {3, W)) + {minors(3,Y)) + {minor s {3, Z)). 

It is also easy to check the dimensions of subspace varieties. A con- 
venient tool is to use the Kempf-Weyman desingularization via vector 
bundles. Let Si denote the canonical (subspace) rank i vector bundle 
over the Grassmannian Gr(z,n). The desingularization is 

F{Sp ® 5g ® Sr) X Gr(p, 3) x Gr(g, 3) x Gr(r, 3) Subp,^,^ . 
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In particular, we have 

dim(Subp,g,r) = VQ.'^ — 1 + p(3 — p) + ^(3 — q) + r(3 — r). 
We computed the degrees of each subspace variety using Macaulay 2: 



variety 


Sub2,3,3 


Sub2,2,3 


Sub2,2,2 


dimension 


19 


15 


13 


CO dimension 


7 


11 


13 


degree 


36 


306 


783 



Another description of /(Subp^, ,.) in Representation Theoretic lan- 
guage will allow us to compare the equations here with any other G- 
invariant sets of equations, no matter how they are presented. An- 
other way to state jLW07l Theorem 3.1] is that for each integer c?, 
/(Subp,g,r-)d consists of all representations S\A ® S^B (g) S^C* with par- 
titions X,fi,h'\-d such that either |A| > p, \fi\ > q or |z/| > r. 

In fact, the ideals of subspace varieties are generated in the minimal 
degree d possible. To save space, we write SxS^S^, for the representation 
SxA (g) S^B (g) S^C*, and in place of 

/(Sub2,3,3) = {^^S^ © K'S2lS21 © A'^^'A') 
/(Sub223) = /(Sub233) + {821^ S21 © ^ ^) 
/(Sub2,2,2) = /(Sub2,2,3) + (<S'2lS'21 A^) 

Finally, comparing to Nurmiev's list |NurOOa] . the variety Sub2,3,3 
corresponds to nilpotent orbit 9 (and 9' and 9" correspond to permu- 
tations of Sub2,3,3). The variety Sub2,2,3 corresponds to nilpotent orbit 
number 17 (and 17' and 17" for permutations). Sub2,2,2 is also equal 
to the secant variety of a Segre product, (T2(Seg(P^ x x P^)) and 
corresponds to nilpotent orbit number 20 on Nurmiev's list. 

4.2. P-Rank varieties. P-Rank varieties are defined by considering 
the three images of the projections of a tensor onto two of the factors 
and restricting the rank of points in the image. In particular (see 
|Lanl2l §7.2.2]) Rank^ is the projectivization of the set 

{T eV \ rank(T(A)) < r}, 

where if T has slices (in the A*-direction) Wi, W2 and W3, T{A) is 
the matrix Wi + W2 + W3. Rank^ and RankJ^ are defined similarly. 
Let P-Rank^'^'^ denote the projectivization of the set of tensors T with 
P-Rank(T) < {p,q,r). Equivalently, 

P-RanF'''''' = Rank^ n Rank^j n Rank^ . 

It is easy to check that P-Rank^''?''' is SL(A) x SL(5) x SL(C)-invariant. 
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While P-Rank varieties can be considered in arbitrary dimensions, 
we restrict to the case that A, B and C are 3-dimensionaL 

Landsberg points out that Rank^ is usually far from irreducible. In 
particular, there are at least two subspace varieties in Rank^ 

Sub3,3,2 U Sub3,2,3 C Rauk^ = P-Rank^'^'^ . 

In fact, we will see later that there is yet another component. Similarly 

Sub3,3,2 U Sub3,2,3 C P-Rauk^'^'^ 

Sub3,3,2 U Sub2,3,3 C P-Rauk^'^'^ 

imply a third containment Sub3^3.2 U Sub2,2,3 C P-Rank^'^'^. 

Moreover the 3-way intersection certainly contains the following 

Sub3,2,2 U Sub2,3,2 U Sub2,2,3 C P-Rauk^'^'^ . 

But, in fact, all of the inclusions above are strict containments. 

In Section [7] we consider the poset of orbit closures in P-Rank^'^'^. 
This will allow us to show that P-Rank^'^'^ is irreducible and corre- 
sponds to the orbit closure consisting of tensors whose projections 
T{A), T{B) and T(C) are skew-symmetrizable 3x3 matrices. One can 
also show that the irreducible components of P-Rank^'^'^ are Sub3^3.2 

2 2 2 / \ \ 3 S 2 

and P-Rank ' ' (which contains Sub2, 2,3)- Further, we'll see that P-Rank ' ' 
consists of four distinct components, namely the two subspace varieties 
Sub2,3,3 U Sub3,2,3, the trifocal variety X, and P-Rank^'^'^. 

5. Symbolic computations using Representation Theory 

In this section we compute the trifocal ideal I{X) up to degree 9, 
and then find the minimal generators among those polynomials. 

A systematic algorithm to compute all polynomials in low degree 
in the ideal of a G- variety is described in |LM04j . We carry out this 
algorithm for the trifocal variety. In a word, the test is to decompose 
the ambient coordinate ring as a G-module and check every module of 
equations in low degree for membership via representation theory. 

Recall that the trifocal variety X is the closure of a G-orbit in PV^, 
with G = SL{A) X SL{B) x SL(C). This allows us to use tools from 
representation theory to compute and understand the ideal of X. 

The coordinate ring of PV" has a G-module decomposition in each 
degree as 

S'^V* = (SxA (g) S^B ® S,C*) ® C™^-''-% 

X,fj,,u\-d 

where SxA ® S^^B S^C* is an isotypic module associated to partitions 
A,/i, z/ of d, and rrix^^^y is the multiplicity of that isotypic component. 
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|LM04t Proposition 4.1]. One advantage to considering polynomials in 
modules is that we can compare different sets of polynomials no matter 
in what basis they are presented and determine if they are the same. 

Even more useful is the following. Because we have a reductive group 
acting, we have the following splitting as G-modules 

k[V] = I{X)®k[V]/I{X). 

To determine I{X), then we just need to determine which irreducible 
G-modules are in I{X). 

Here is a short synopsis of the algorithm in |LM04j . Suppose M is 
an irreducible G-module in 5"^*. To determine whether M C I{X) 
or M C k[V]/I{X), it suffices to check whether a random point on X 
vanishes on the highest weight vector of M. Random points of X may 
be constructed by acting on a normal form by random elements of G. 
If M is an isotypic component of S*V* that occurs with multiplicity 
m, we first construct a basis of the highest weight space C"^ of M, by a 
straightforward construction involving Young symmetrizers. Then we 
select m random points from X and construct an m x m matrix whose 
i, j-entry is the ith point evaluated on jth basis vector. The kernel of 
this matrix tells the linear subspace of M that is in the ideal of X. 

Of course because we use random points, we should then re-verify 
all vanishing results symbolically to rule out false positives (there are 
no false negatives because non-vanishing at random points of X im- 
plies non- vanishing on X.) We did these extensive computations in 
Maple, and we have provided a sample of our code in the ancillary files 
accompanying the arXiv version of this article. 

We can further determine which among the polynomial modules we 
find are actually minimal generators. Suppose the degree d piece of 
the ideal, I{X)d, is known and has been input into Macaulay2 as I. 
Then in the next step of the Landsberg-Manivel algorithm, for each 
new highest weight vector / (a polynomial of degree d + 1) check if 
/ G {I{X)d) by quickly computing fy„I. The module {G.f} associated 
to the highest weight vector / is in {I{X)d) if and only if f °/„I is zero. 

We tabulate the results of this test applied to the trifocal variety 
below. Again, to save space we write SxSf^S,, in place of SxA (g) S^B (g) 
SyC* . The trifocal variety has an ©2 symmetry permuting the A* and 
B* factors. To further save space, where appropriate, we will write 
&2-{SxS^i)Sy to indicate the sum S\S^Sy -\- S ^S\Sy. 

Proposition 5.1. Let X denote the trifocal variety in ¥V and let 
denote the space of minimal generators in degree d of I{X). There are 
10 minimal generators in degree 3, 81 in degree 5, and 1980 in degree 
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6. The G-module structure of the minimal generators is as follows. 

Ms = A'A''5^ 

= (5'22l5'22l)(5'3ii © 5*221) 
Me = (©2- (5222^33) (^33 © ^411)) © 62. (^33^32l)^321 

© (62-(>S'33S'4ii) © S'ssS'sa) 5*222, 
and there are no other minimal generators in degree < 9. 

By recording the dimension of all modules that occur, this computes 
the first nine values of the Hilbert function of k[V]/ I{X): 

27, 378, 3644, 27135, 166050, 865860, 3942162, 15966072, 58409126. 

During our tests in Maple, we computed a basis of each module and 
provide these equations in the ancillary files accompanying the arXiv 
version of this paper. 

We relate some of the polynomials we found to the known polynomi- 
als in |AT10j . Landsberg proves that Rank^ is the zero-set of S'^^^A © 
/^^^B^f^^^C* |Lanl2i Proposition 7.2.2.2]. One can also phrase the 
condition that T e Rank^ as the requirement that the matrix T[A) of 
linear forms from A has rank not exceeding r. If A is m-dimensional, 
a basis of the module S'^^^A © f{^^B © f{^^C is given as follows. 
Consider the slices T^,...,T™ of T in the A-direction, and dummy 
variables Xi,...,Xm- The condition that rank(^^j^ XjT*) < r is the 
condition that all coefficients (on the Xj) of the (r + 1) x (r + 1) minors 
of the matrix Y^iLi vanish. So a basis of S^^^A © f^^^B © f^^^C 
is given by the (polynomial) coefficients of these minors. 

Recall that a normal form for a point T on the trifocal variety has 

/O C2 cA 

T(C) ^ C3 . 
Vci 0/ 

And this matrix clearly has rank < 2. 

The above discussion implies that X C Rank^, and the module 

Ms := /^A® /^B © S^C* 

is in the trifocal ideal. These equations were also identified in |AT10] . 

We now describe the two modules in M5. The highest weight vectors 
are polynomials with (respectively) 104 and 244 monomials and multi 
degrees [(2,2,1), (2,2,1),(3,1,1)] and [(2,2,1),(2,2,1),(2,2,1)], in the ring 

k[aii, . . . , 033, 611, . . . , 633, Cii, . . . , C33]. 
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Here we are using aij to denote Tjji, bij = Tjj2, and Cij = Tij^. Typical 
terms of the highest weight vectors are 

&3lC22ai3&12ail + &3iaL^23Cll - &3lC2ia?2^13 

+ &22C3iai3ai2&ii - &22C3iaii6i3ai2 - a32C22fcnai3 ■ ■ ■ 

and 

a2i&12&33Cl2 - 2ai2a336ii622C21 - a2i&12&13C32 

+ Cfl2023^11^2lC32 + 021^12^33 • • • 5 

respectively. The basis of 5*221 >S'22i5'22i consists of 27 polynomials which 
are all equal after a change of indices. All the coefficients come from 
the set {— 5, — 2, — 1, 1, 2, 4}. The basis of 5'22i5'22i5'3ii consists of 54 
polynomials which are of two different types having either 104 or 64 
monomials and coefficients in the set {—1,1}. The polynomials them- 
selves can be downloaded from the web as mentioned above. 

For Mg we can give a similar description. 5*222 'S'33 6*33 and 5*222 'S'33 6*411 
and 5335222^411 are all 100-dimensional, each with a basis consisting of 
polynomials that have between 66 and 666 monomials and small (abso- 
lute value no greater than 4) integer coefficients. Similarly 53215335321 
is 640-dimensional with a basis consisting of polynomials that have be- 
tween 60 and 732 monomials and small integer coefficients. The full set 
of polynomials is available with the ancillary files on the arXiv version 
of the paper. 

After computing /(X)^ for small d, we computed a Grobner basis 
of J = (M3 -|- M5 + Mq) in Macaulay2. Surprisingly, this compu- 
tation finished in a few minutes — it actually took longer to load 
the polynomials into M2 than it took to compute the Grobner ba- 
sis. We were also able to compute a Grobner basis of ©3.M3 = 
{f\f\S^) © {/^S^/^) © (S^/^/^), the 30 cubic equations defining the 
rank variety P-Rank^'^'^. We record the results of these computations: 

Proposition 5.2. Let X denote the trifocal variety and let Ma denote 
the space of minimal generators in I{X)d- The following computations 
done over Q hold: 

deg(V(63-M3)) = 1035 and codim(V(63-M3)) = 10. 

deg(V(M3 + Mg + Me)) = 297 and codim(V(M3 + M5 + Me)) = 8. 

Proof. The proof is by computations in Macaulay2 |GS02j that we pro- 
vide with the ancillary files in the arXiv version of the paper. □ 
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Though we don't need it for our proof, we were also able to compute 
the Hilbert polynomial of J = (M3 + M5 + Mq): 

69P5 - 423F6 + 882P7 - 204P8 - 2565P9 

+ 575IP10 - 6I29P11 + 3402P12 - 783Pi3 + IOOP14 

- 525Pi5 + 1038Pi6 - 909Pi7 + 297Pi8, 

where we use the variables Pj following the standard normalization 
used in Macaulay2 to describe the Hilbert Polynomial. 

6. Numerical Algebraic Geometry: Bertini 

In Numerical algebraic geometry, and specifically using the program 
Bertini, one can compute numerical primary decompositions of ideals 
if the number of equations and the degrees of those equations are rela- 
tively small. In contrast to Grobncr basis computations where typically 
more equations is better, in numerical algebraic geometry it is better 
to start with the lowest degree and lowest number of equations that 
one can understand. Then one can try to compute a numerical pri- 
mary decomposition and attempt to work by other means to obtain a 
geometric description of the components indicated by Bertini. 

Following this philosophy, we started with the 10 equations in degree 
3 given by the complete vanishing of 

det(xiZi + X2Z2 + X3Z3), 

which define Rank^. Recall that {Zi \ Z2 \ Z3) is the flattening of 
a tensor in the C-direction. These are specifically the 10 equations 
defining the module M3; that is, a basis for M^. 

After about 6.5 hours of computational time on 2 processors (and 
some help from J. Hauenstein getting the initial parameters right), or 
just under 10 minutes on Hauenstein's cluster, Bertini succeeded to 
compute the following numerical decomposition. 

Computation 6.1. Let Mz denote the 10 coefficients (in xi,X2,X3) of 
the cubic det(xiZi + X2Z2 + X3Z3). Up to the numerical precision of 

Bertini, the zero set of M3 has precisely 4 components: 

In codimension 7 there are 2 components, each of degree 36. 
In codimension 8 there is 1 component of degree 297. 
In codimension 10 there is 1 component of degree 1035. 

It is not too hard to show that the two components in codim 7 are 
the subspace varieties 

Sub3,2,3 U Sub2,3,3 
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This is because they have the correct dimension, M3 is in both ideals, 
and their ideals are generated (respectively) by 

{f\f\S') © (^'A'A') and (A'A''^') © (A'-^'A')- 
We know that the trifocal tensor variety is in the zero set and has 
codimension 8. It is not contained in either subspace variety, so we 
may conclude that X corresponds to the codimension 8 component in 
the numerical decomposition. We also learn that X has degree 297. 
The variety P-Rank^'^'^ must correspond to the codimension 10 com- 



ponent, which we prove in Proposition 7.2 below. In addition, this 
Bertini computation also tells that P-Rank '^'^ has degree 1035. 



7. NURMIEV'S CLASSIFICATION OF ORBIT CLOSURES AND THE 
PROOF OF THE MAIN THEOREM 

The orbits of SL(3)^^ acting on C'^ (g) ® have been classified by 
Nurmiev [NurOObj . who also computed the closure of most orbits. 

Using Nurmiev's list of normal forms, we can quickly check which 
orbits are contained in V{Ms) by taking a parameterized representative 
for each orbit (normal form) and evaluating the polynomials in M3 
on that representative. This can be carried out in a straightforward 
manner in any computer algebra system. 

The following orbits from Nurmiev's list of nilpotent orbits [NurOOat 
Table 4] are in V(M3): 9, 9', 11", 12, 12', 13, 13', 14, 15, 15', 16, 16', 
17, 17', 17", 18, 18', 18", 19, 19', 19", 20 (= 20' = 20 "), 21, 2 1', 21", 
22, 22', 23, 23', 23", 24 (= 24' = 24"), 25 (= 0), see [NurOObj for an 
explanation of the notation used. 

After considering the nilpotent orbits, we must also consider the 
semi-simple orbits together with their nilpotent parts. Among these 
orbits, our tests found that only one semi-simple orbit, namely the one 
corresponding to Nurmiev's fourth family, is in V{M^). In our notation 
we may represent this normal form as 

F = A(ai ® &2 ® C3 + 02 ® &3 ® ci + 03 ® 61 ® C2 

- ai ® ^3 ® C2 - a2 ® 61 ® C3 - 03 (g) 62 ® ci), 

for any scalar A 7^ 0, but over the complex numbers this scalar may be 
absorbed. 

Remark 7.1. As a matrix with linear entries either in A*, B* ot C a. 
normal form for F is always of the form 

X —y^ 
—X z \ . 

y -z 
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Namely this orbit corresponds to the skew symmetric matrices. More- 
over, since this matrix is skew-symmetric, it always has even rank, and 
thus we find that F G P-Rank^'^'^ with no computation necessary. 



Next we consider the closures of all the nilpotent orbits in [NurOOat 
Table 4]. Our inclusion poset diagram in Figure |2] is enlightening. 
For all arrows except for those emanating from F, the diagram is a 
restatement of results in [NurOOaj . Namely, if orbit Q is in the closure 
of orbit P (as indicated by Nurmiev's table) and there isn't already a 
directed path from P to Q we draw an arrow from P to Q. 

We then consider all orbits that are in the zero set of M3 for all 
permutations of A,B and C. We write this zero set in shorthand as 
V(©3.M3). Since F is a zero of ©3.M3, every orbit in its closure must 
also be in this zero set. A straightforward computation shows that 
these orbits in Vl&^.M^) are numbers 17-21, 23, 24 (and all of their 
primed versions). Of course this does not imply that these orbits are 
actually in the closure of F. However, it is enough to check that orbits 
17, 18 (and their primed versions) are in the closure of F. 

Proposition 7.2. The zero setV{&3.M^) (which equals F-Kank^''^''^ by 
definition) is irreducible and is the closure of the orbit F above. 

Moreover, the orbit associated to the normal form F is not contained 
in any of the other orbit closures in Figure 

Proof. This proof is entirely computational, but since we did not find 
it in the literature, we provide the computations here. 

Orbit closures consist of one orbit of the top dimension along with 
other orbits of lower dimension. So we conclude by counting dimensions 
that none of the orbits 9, 9', 11", 12, 12', 13, 13', 14, 15 or 15' are in 
the closure of F. Later we will show that 17, 18 and all their primed 
versions are in the closure of F. 

We claim that none of the orbits of higher dimension (9, 9', 11", 12, 
12', 13, 13') contain F in their closure. Consider the normal form of F 







-X 



X 





and fiatten it to the matrix 
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~z 





which, for general choices of x, y, z has full rank. The other slices have 
similar format, and this shows that F is not contained in either of the 
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subspacc varieties Sub2,3,3 or Sub3,2,3 (the closures of orbits 9 and 9"). 
This also implies that F is not in the closure of any orbit contained in 
the closure of 9 or 9". 

To show that F is not contained in X, we could demonstrate a 
polynomial in I{X) that does not vanish on F. We already noted that 
the skew-symmetric matrices in F has rank 2 or less and thus vanishes 
on all polynomials in M3. The ideal I{X) has no minimal generators in 
degree 4, so we must start to consider polynomials in degree 5 or higher. 
Here we notice by direct computation that neither of the modules in 
M5 vanish on F, separating F from X. 

Another way to conclude Rank^ ^ V{M^), without computation, is 
to consider the degree 5 piece of the ideal generated by M-^ = f^S^. 
The Fieri formula gives that every module of I{M^) in degree 5 must 
have a partition in the first position whose first part is at least 3. On 
the other hand, the module «S'22i 5*221 5'22i in ^5 fails this property So 
it cannot be in the ideal /(M3) of Rank^, and thus Rank^ (jL X. 

Any orbit in V{M^) is either in the closure of F or in the closure of 
9, 9' or 11". Moreover, because F is not in the closure of 9, 9' or 11", 
its closure must be an irreducible component of V{M^). 

Since <33.M3 manifestly has ©3 symmetry, so does its zero-set. Thus 
to prove that V(S3.M3) is irreducible, wc need to show that orbits 17, 
18 and their primed versions are contained in the closure of F. This will 
imply irreducibility of V(63.M3) because every orbit contained there 
is in the closure of a single orbit. 

Since F is symmetric with respect to permutation by ©3, it suffices 
to prove that orbits 17 and 18 are contained in the closure of F. To do 
this we exhibit a sequence of points in the orbit of F that converge to 
each orbit. We prefer to work with the normal forms both as tensors 
and as matrices of linear forms. The group operations allowed are row 
and column operations as well as general linear changes of coordinates 
on each linear form appearing. 

Nurmiev's orbit 17 has normal form given by the matrix 

ai 

ai , 
a2 / 

or the tensor 

oi ® {hi ®C2 + h2® ci) + a2® {hi (8) C3 63 ci). 

Replacing hi with —hi we obtain the tensor 

oi (8) {—hi ®C2 + h2® ci) 02 (8) {—hi (8) C3 63 Ci) 
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= -ai (8) 61 ® C2 + ai (g) 62 ® ci) + -a2 (8 61 ® C3 + 02 ® 63 ® ci), 
which corresponds to the (skew symmetric) matrix 

— ai —a2\ 

01 . 

02 / 

This matrix is a hmit of the following matrices in the orbit of F 

— oi —02 

ai Zn 
a2 -Zn 

where 2;„ — )■ 0. 

For the orbit 18, we consider the following representative as a tensor 

ai (g) (61 ® Ci + 62 ® C2) + 02 ® (&i ® C2 + 62 ® cs) 

which corresponds to the matrix 

Oi a2 \ 

Oi 02 . 

0/ 

By cycling rows 1, 2 and 3 we at least have a matrix with diagonal, 

tti 

0. 

01 a2 J 

Now we work with the following matrix normal form for F: 

X -y\ 
—X z 
y -z oj 

which corresponds to the tensor 

F = z®h2®Ci+y®hz®ci+x®hi®C2—z®h'i®C2—y®hi®C'i—x®h2®ci. 

First we set x = ai and —y = 02- Considering a limit of such tensors 
gives a tensor in the closure with 62 = corresponding to the matrix 

ai a2\ 
0. 
— a2 —z J 

Multiply the 3rd row by to get another matrix in the same orbit 

(0 oi a2\ 
. 
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Finally we set z = to yield a matrix in the orbit 18. □ 

The above discussion provides the following effective test for a given 
tensor T to be a trifocal tensor. Namely the orbit of trifocal tensors 
is precisely the G-invariant set of tensors with P-Rank(T) = (3, 3, 2) 
or some permutation thereof, and F-Rank(T) = (3,3,3). These two 
conditions contain both vanishing and non-vanishing conditions. We 
phrase this as an algorithm to determine whether a given tensor is a 
trifocal tensor. 

Algorithm 1. 

Input: A tensor T G (g) (g) 

• Replace T by a change of coordinates ( either arbitrary or ran- 
dom) from GL{A) x GL{B) x GL{C) applied to T. 

• Compute the projections T{A),T{B),T{C) . Is P-Rank(T) = 
(3,3,2) (or some permutation) and no less? 

NO: stop, T is not a trifocal tensor. 
YES: continue 

• Compute all 3 flattening s. Is F-Rank(T) = (3, 3, 3) and no less? 

NO: stop, T is not a trifocal tensor. 
YES: T is a trifocal tensor. 

If one uses arbitrary changes of coordinates (with parameters) the 
conclusions of Algorithm 1 hold without modification. However, it may 
be difficult to perform the tests. If one uses random changes of coor- 
dinates. Algorithm 1 will go quickly, and the negative conclusions are 
sure, but the positive conclusions will hold only with high probability. 

This test is effective because it involves computing the ranks of three 
3x3 matrices and the ranks of three 3x9 matrices. This test is similar 
in spirit to the results in |AT10t Section 4]. 



Proposition |7.2| yields the following geometric statement. 

Proposition 7.3. Let X denote the trifocal variety (the closure of orbit 
11"). Then the irreducible decomposition ofV{M^) = P-Rank^'^'^ is 

ViMs) = Sub2,3,3 U Sub3,2,3 UX U P-Rank^'^'^ . 

Proof. To see that Vi^M^,) contains the four listed components, just 
construct a normal form for each and notice that the associated matrix 
in linear forms has rank < 3. To see that these are the only compo- 
nents, look at the orbit closure diagram in Figure |2| which displays 



all orbits in V{Ms) and is justified by |NurOOa] and Proposition 7.2 
Notice that there are 4 sources in the directed graph representing the 
poset and these correspond to the only irreducible components in the 
decomposition. □ 
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Nurmiev's table includes the dimension of the stabilizer of each orbit, 
which tells the codimension of each of the components: codim(Sub2,3,3) = 
7, codim(X) = 8, codim(P-Rank ' ' ) = 10. Nurmiev's computation is 
confirmed by the computation done in Bertini; however, Bertini tells 
us a bit more, namely the degree of each component. 

Proposition 7.4. The zero set V(M3 + M5 + Mq) is irreducible and 
agrees with X set-theoretically. 

Proof. We need to show that when we intersect V(M3) with V(M5+Mg) 
that all of the orbits that remain are actually in the trifocal variety. It 
suffices to show that orbits 17, 17', 18', 18" are not in VIM^+Mq). This 
is because by considering the orbit closure poset diagram in Figure [2| 
these orbits are contained in all other orbits in V(M3) that are not in 
the trifocal variety X, so if they are not in V(M5 + Mg), then no other 
G-orbit in V^Ms) outside of X is in V(M3 + M5 + Mg). 

By direct computation, we find that the module 5'33 5*222 5'4ii does not 
vanish on orbit 17, the module 5'222'S'33S'4ii does not vanish on orbit 17', 
the module 5'33S'33S'222 does not vanish on 18', and 5*222 'S'33 5*33 does not 
vanish on 18" . On the other hand, each of these modules are in Mg. □ 

8. The ideal J is prime 

Let J = (M3 + M5 + Mg), where are the minimal generators of 
I{X) in degree d. The trifocal variety X is irreducible because it is a 
parameterized variety. The fact that the zero set V{J) is irreducible 



and equals X is the content of Proposition 7.4 So I{X) and J agree up 
to radical. It remains to check that there are no embedded components. 

The classification of G-orbits in V also yields a classification of min- 
imal G-invariant prime ideals. To every orbit is associated the prime 
ideal of its orbit closure. 

Remark 8.1. Here we also use the fact that if G is a connected group, 
and J is a G-stable ideal, then the minimal primes in any primary 
decomposition of J are G-stable. This essentially follows from the fact 
that if J = CiiQi is a primary decomposition with primary ideals Qi 
associated to primes Pi, then gj = J = (ligQi for any g E G and this 
action must permute the Pi by the uniqueness of minimal primes. But 
since G is connected, this permutation must be trivial. 

The poset in Figure [2] shows that the minimal prime ideals that 
contain I{X) are those corresponding to orbits 14, 15, and 15'. Let 
Pi4, Pi5 and P15/ denote the corresponding prime ideals. Then we 
must have J C P14 fl P15 fl P15/. On the other hand, we know that y/j 
is prime and equals I{X). So a primary decomposition of J is of the 
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form J = /(X)n(5i4n(5i5n(5i5, for some primary ideals Qi associated 
to the primes Pi. We will show that the multiplicity of each Qi with 
respect to Pi is zero. 

If we show this, we don't have to consider possible embedded com- 
ponents coming from the other orbits in the closure of X because these 
ideals contain P14, P15 and P15'. Moreover, since X and J have an 
©2 symmetry, if we show that the P15 does not occur in the primary 
decomposition, then neither does P15'. 

We will use a basic fact from commutative algebra. We found |BV88t 
Theorem 12.1] a useful formulation for understanding this type of test. 

Proposition 8.2. |AM69l Proposition 4.7] Let a he a decomposable 
ideal in a ring A, let a = n^_;^qj be a minimal primary decomposition 
and let pi be the prime ideal associated to the primary ideal qj. Then 

ur=iPi = {x e A \ {a: x) ^ a}. 

In particular, if the zero ideal is decomposable, the set D of zero- divisors 
of A is the union of the prime ideals belonging to 0. 

We also have the following well-known fact (see for example |Eis99j ). 

Proposition 8.3. Let R = k[xo, . . . Xn], let J be an ideal in R and 
suppose f & R has degree d and is not a zero-divisor in R/ J . Then we 
have the following identity of Hilbert series: 

il-t')Hn/At)=Hn/ij+f)it). 

Proof. This is completely standard, but we recall the proof here for 
the reader's convenience and because it elucidates the ideas we will use 
later. 

If / is not a zero divisor, the following sequence is exact 

(1) — y{R/J){-d)^R/J — yR/{J + f) — ^0. 

Since = t'^H(^R/j){t), the result follows from the additivity 

of Hilbert series. □ 

Remark 8.4. If / is actually a zero-divisor of R/J, then in some degree 
t', the graded version of the sequence ([T]) will have a kernel K larger 
than expected. This will force the inequality in 

i"^H(^ji/j){t') = < Hxit'). 

In this case we will have 

i"^H(^ji/j){t') - Hji/j{t') < HK{t') - Hji/j{t') = HB./(j+f){t'), 

which implies that 

{l-t"')Hn,j{t')<Hn/ij^f^{t'). 
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The previous results allow for the following test. Since the zero- 
divisors of J correspond to the union of prime ideals Pi that contain J, 
we can select one f E Pi of degree d which vanishes on the subvaricty 
V(Pj) C V(J) but does not vanish on V(J). If we show that (1 — 
t''')Hji/j{t) = Hji/(^j^f){t), then / is not a zero-divisor of R/J. This 
would show that could not have been a prime ideal associated to J. 

We provide the results of this computational test with the prime 
ideals P15 and P14. We wanted to check if a map had a kernel, so 
we worked over characteristic 101. Non-vanishing modulo a prime p 
implies non- vanishing in characteristic 0. 

In Macaulay 2 we computed a Grobner basis of J = (M3 -|- M5 -|- Mq) 
in about 30 seconds. The Poincare polynomial Pj of P/ J is 

Pj = l-10r^-8ir^-1605r^+18117r^-77517r^+192794r''-315792r 

+350676r^^-243572r^V48438T^^+116883T^^-175239r^^+140238T^^ 

- 75330T^^ + 27954T^^ - 6912T^'^ + 1026T^° - 69T^^. 

For the prime ideal P15 we constructed slices in the P-direction 
Yi,Y2,Y3, computed det(xiyi -|- X2Y2 + xsYs) = and selected the 
polynomial / as the coefficient of . Precisely, 

(an ai2 aiaX 
^11 bi2 613 ■ 
Cll C12 Ci3 J 

This polynomial / vanishes on P-Rank^'^'^ and V(Pi5) but not on 
P-Rank^'^'^, and thus not on X. 

Computing a Grobner basis of J -|- (/) took about 10 hours on a 
server that allowed us to use 16GB of RAM and up to 8 Intel(R) 
Xeon(R) CPU X5460 3.16GHz processors. The Poincare polynomial 
Pf of P/( J + /) is 

P/ = 1 - lli^ - Slt^ - 1595t^ + ISmf - 77436i^ + 194399^*^ 
- 333909i^° + 428193^^^ - 436366^^^ + 364230^^^ - 233793^^^ 
+ 68333i^^ + 91800^^^ - 192213i^^ + 203193i^^ - 147150^^'' 

+ 76356f^ - 28023i2^ + 6912^^^ - 1026i2=^ + 69^^^ 

Now it is easy to check that (1 —t^)Pj = Pf, which implies 

and thus / is not a zero-divisor of R/ J. The prime P15 is thus not an 
embedded prime of J. By the 62 symmetry of J, we conclude that 
P15/ is also not an embedded prime of J. 
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For the prime ideal P14, the module 5'225'2i 15*211 vanishes on V^Pu) 
but not on X. We select the highest weight polynomial g for our test: 

9 = ai3a2i&i2C2i - ai3a2iCi2&2i + 3a23ai2&iiC2i + £13021012621- 

613021012021 — 3O23O12C1162I + '3-230'll622Cll — a.23Q-llC226ll + 
2022021^^13611 — C22ai3'J^ll621 — 2622O21O13C11 + 622CI21C13O11 + 
622^13011021 - 022^21613011 + 023011^22611 + 2623021012011- 
623O21O12O11 — 623O12O11O21 — O13612O21 + 613O12O21 — 
O23622O11 + 6230220^;^ — O22O21613O11 — 2O23O21O12611 + 
O23O21612O11 + O23O12O11621 - 623O11O22O11 + O23O21612O11- 
O23O21O12611 + O22O21O13611 - 3O22O13611O21 + 3O22O13O11621- 

2O21O23612O11 + 2O21O22613O11 + 2621O23O12O11 - 2621O22O13O11. 

Computing a Grobner basis of J + (g) took about 45 hours to finish 
on a server that allowed us to use 16GB of RAM and up to 8 processors. 
The Poincare polynomial Pg of R/ {J + g) is 

Pg = l- lOt^ -t^ - 81t^ - 1605t^ + 18127t^ - 77517t^ + 192875t^ 

- 314187t^° + 332559t^^ - 166055t^^ - 144356t^^ + 432675t^^ 

- 525915t^^ + 383810t^^ - 123768t^^ - 88929t^^ + 168327t^^ 

- 139212^20 + 75261^2^ - 27954^^2 + 6912^^3 - 1026t2^ + 69^^^ 

It is again a simple check that (1 — t^)Pj = Pg, which implies 

As before, g is not a zero-divisor of R/J, and the prime P14 is not an 
embedded prime of J. 

We have shown the following 

Theorem 8.5. The ideal J = (M3 + M5 + Mq) is prime. 



Proof. By Proposition 7.4 we know that \/j = I{X). By Proposition 



5.2, we know that the degree of the top dimensional component of J is 



297, counted with multiplicity. By Computation 6J^, we know that the 
degree of X is 297. So we know that in a primary decomposition of J, 
/(X) occurs with multiplicity 1. It only remains to rule out embedded 
primes. By the above discussion, if we have a primary decomposition of 
the form J = I{X) fl Q14 H Qis H Qis', where the Qi are primary ideals 
associated to the prime ideals Pi, then we showed that their multiplicity 
must be zero. So J = I{X), and in particular J is prime. □ 



26 CHRIS AHOLT-*, LUKE CEDING^ 



This completes the proof of Theorem 1.1 We conjecture that a 



similar calculation will work to show that the ideal of the orbit closure 
associated to F is minimally generated by 63. M3. 
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